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EXECUTIVE  SUMMARY 


Issue  and  Overview 

The  purpose  of  this  report  is  to  evaluate  the  importance  of  trainee  characteristics  to  learning  in  Special 
Operations  Forces  (SOF)  initial  acquisition  foreign  language  training  (1AT).  This  report  focuses  on 
trainees’  general  cognitive  ability  and  motivation  to  learn  as  potential  predictors  of  achievement  during 
1AT.  The  following  research  questions  are  examined  in  this  report: 

•  Do  trainee  general  cognitive  ability  and  motivation  to  learn  predict  learner  achievement  during 
1AT? 

•  Do  trainee  general  cognitive  ability  and  motivation  to  learn  independently  or  jointly  predict 
achievement  during  1AT? 

Findings 

Do  trainee  general  cognitive  ability  and  motivation  to  learn  predict  learner  achievement  during  IAT? 

•  Trainees’  general  cognitive  ability  (as  assessed  using  the  Armed  Forces  Qualification  Test)  was 
found  to  predict  achievement  on  assessments  of  both  listening  and  reading  skills.  Trainees  higher 
in  cognitive  ability  showed  greater  achievement  than  those  lower  in  cognitive  ability. 

•  Motivation  to  learn  showed  an  inconsistent  relationship  with  achievement.  After  accounting  for 
training  language,  more  highly  motivated  trainees  showed  greater  listening  achievement  than  less 
motivated  trainees.  This  relationship  was  not  found  for  reading  achievement. 

Do  trainee  cognitive  ability  and  motivation  to  learn  independently  or  jointly  predict  achievement  during 
IAT? 

•  Trainee  cognitive  ability  and  motivation  to  learn  showed  independent  (or  additive)  relationships 
with  achievement.  We  did  not  find  evidence  of  a  higher-order  interaction  between  these  two 
trainee  characteristics  in  the  prediction  of  trainee  achievement  during  IAT. 

Conclusions 

•  Trainee  characteristics  appear  to  be  predictive  of  learning  performance  during  IAT.  Specifically, 
trainee  cognitive  ability  and,  to  a  lesser  extent,  motivation  to  learn  are  useful  indicators  of  a 
trainee’s  performance  on  listening  and  reading  skills  assessments  during  training. 

o  AFQT  scores  may  be  useful  for  placing  trainees  into  specific  training  languages,  such 
that  higher  ability  learners  could  be  identified  and  placed  into  more  difficult  training 
languages.  Placing  trainees  with  greater  learning  potential  into  more  difficult  languages 
could  improve  learning  outcomes  (e.g.,  skill  acquisition)  and  training  success  in  those 
languages. 

This  project  was  conducted  by  SWA  Consulting  Inc.  under  a  subcontract  with  CACI-WGI,  Inc. 
(Subcontract#  B1 1-1 14482;  Prime#  H92222-10-D-00 17/0007;  Sub-CLIN  0003AB).  For  questions  or 
more  information  about  the  SOFLO  and  this  project,  please  contact  Mr.  Jack  Donnelly 
(john.donnelly@socom.mil).  For  specific  questions  related  to  data  collection  or  this  report,  please  contact 
Dr.  Eric  A.  Surface  (esurface@swa-consulting.com)  or  Dr.  Reanna  Poncheri  Harman  (rpharman@swa- 
consulting.com). 
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REPORT  DETAILS 

This  report  was  produced  as  part  of  the  Special  Operations  Forces  Language  Office  (SOFLO, 
USSOCOM)  training  analysis  support  initiative.  The  focus  of  this  report  is  to  document  the  relationships 
between  Special  Operations  Forces  (SOF)  trainees’  general  cognitive  ability  (as  assessed  by  the  Armed 
Forces  Qualification  Test),  motivation  to  learn,  and  achievement  during  initial  acquisition  foreign 
language  training  (IAT).  This  report  addressed  the  following  research  questions: 

•  Do  trainee  general  cognitive  ability  and  motivation  to  learn  predict  learner  achievement  during 
1AT? 

•  Do  trainee  general  cognitive  ability  and  motivation  to  learn  independently  or  jointly  predict 
achievement  during  1AT? 


Sample 

The  sample  consisted  of  550  trainees  who  completed  foreign  language  1AT  at  a  U.S.  Army  base  in  the 
southeastern  United  States  between  2005  and  2007.  Training  duration  was  18  weeks  for  Category  1  and  11 
languages  and  24  weeks  for  Category  111  and  IV  languages.  Tests  of  learner  achievement  (also  referred  to 
as  module  tests)  were  administered  on  five  different  occasions  throughout  training.  All  items  on  a  given 
achievement  assessment  covered  material  taught  in  the  previous  training  module.  The  content  of  the 
SOLT  course  curriculum  and  associated  skills  tests  were  professionally  developed  and  standardized 
across  training  languages.  Each  skills  test  included  both  a  listening  and  reading  component.  Each 
component  consisted  of  50  multiple-choice  items  developed  to  assess  trainees’  ability  to  understand  the 
spoken  language  and  read  the  written  language.  Overall  scores  on  each  assessment  was  calculated  as  the 
percentage  of  items  answered  correctly.  Trainees’  scores  on  the  five  listening  assessments  were  averaged 
to  form  an  overall  listening  achievement  measure  (Cronbach’s  a  =  .76).  Reading  assessment  scores  were 
also  averaged  to  form  an  overall  achievement  measure  (a  =  .73). 

Trainees’  general  cognitive  ability  was  assessed  using  the  Armed  Forces  Qualification  Test  (AFQT),  a 
composite  of  four  subtests  on  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB).  AFQT  raw 
scores  were  obtained  from  the  Defense  Manpower  Data  Center. 

Motivation  to  learn  was  assessed  using  a  five-item  self-report  measure  obtained  prior  to  training.  This 
measure  was  developed  for  the  current  study  to  assess  trainees’  motivation  to  learn  the  target  language. 
Sample  items  from  this  scale  are,  “1  would  like  to  learn  as  many  languages  as  possible”  and  “1  am 
motivated  to  perform  well  in  language  training  so  that  we  will  be  able  to  perform  my  missions  more 
effectively.”  Responses  were  provided  on  a  7-point  Likert-type  scale,  ranging  from  1  ( strongly  disagree ) 
to  7  ( strongly  agree).  The  coefficient  alpha  for  this  scale  was  .67. 

Findings — Full  Sample 

Two  multiple  linear  regression  analyses  were  conducted  using  the  full  sample.1  For  the  first  analysis, 
listening  and  reading  overall  scores  were  regressed  onto  trainee  cognitive  ability,  motivation  to  learn,  and 
the  cross-product  (or  interaction  term)  between  these  variables.  Training  language  was  not  entered  as  a 
control  in  this  analysis.  Table  1  presents  the  results.  The  results  indicate  both  cognitive  ability  and 


Means,  standard  deviations,  and  zero-order  correlations  for  all  study  variables  are  provided  in  Appendix  A. 
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motivation  to  learn  positively  predicted  listening  and  reading  achievement  scores.  The  interaction 
between  these  two  variables  was  not  statistically  significant. 

Table  1 .  Regression  of  1AT  Achievement  Test  Scores  onto  Motivation  to  Learn  and  General  Cognitive 
Ability  (n  =  550) 


Step  1 

Step  2 

B 

p 

B 

p 

Listening  Achievement 

Motivation 

.90** 

.12 

.90** 

.12 

Cognitive  Ability 

1.88** 

.25 

1.89** 

.25 

Motivation  X  CA 

- 

- 

.07 

.01 

R2 

.08** 

.08** 

A  R1 

- 

.00 

Reading  Achievement 

Motivation 

.66* 

.11 

.66* 

.11 

Cognitive  Ability 

1.48** 

.24 

1.48** 

.24 

Motivation  X  CA 

- 

- 

-.04 

-.01 

R2 

07** 

07** 

A  R2 

- 

.00 

Note.  CA  =  Cognitive  ability.  B  =  unstandardized  coefficient.  p  =  standardized  coefficient.  Cognitive  Ability  and  Motivation  to 
Learn  were  standardized  for  this  analysis. 

* p<. 05,  **p<. 01. 


Next,  the  aforementioned  regressions  were  repeated  with  training  language  entered  as  a  statistical  control. 
Table  2  presents  these  results.  Controlling  for  training  language,  cognitive  ability  positively  predicted 
listening  and  reading  achievement  scores.  Motivation  to  learn  positively  predicted  listening  achievement 
scores,  but  not  reading  scores.  The  interaction  between  these  two  variables  was  not  statistically 
significant. 

Table  2.  Regression  of  1AT  Achievement  Test  Scores  onto  Motivation  to  Learn  and  General  Cognitive 
Ability  Controlling  for  Training  Language  (n  =  550) 


Step  2a 

Step  3a 

B 

P 

B 

p 

Listening  Achievement 

Motivation 

.67* 

.09 

.67* 

.09 

Cognitive  Ability 

1.64** 

.22 

1.64** 

.22 

Motivation  X  CA 

R2 

23** 

-.00 

23** 

.00 

A  R2 

Reading  Achievement 

.06** 

.00 

Motivation 

.46 

.07 

.46 

.07 

Cognitive  Ability 

1.52** 

.24 

1.52** 

.24 

Motivation  X  CA 

R2 

.19** 

-.02 

.19** 

-.00 

A  R2 

.06** 

.00 

Note.  CA  =  Cognitive  ability.  B  =  unstandardized  coefficient,  p  = 

standardized  coefficient.  Cognitive  Ability  and  Motivation  to 

Learn  were  standardized  for  this  analysis. 
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“Training  language  was  dummy-coded  and  entered  in  Step  1.  Results  for  Step  1  are  not  presented  for  clarity. 

* p  <  .05,  **p<. 01. 

Findings — Modern  Standard  Arabic  Trainees  Only 

To  examine  the  effects  of  cognitive  ability  and  motivation  learn  within  specific  training  languages,  we 
evaluated  the  relationships  of  interest  for  Modem  Standard  Arabic  (MSA)  trainees  (n  =  147).  MSA 
represented  the  largest  single  training  language  in  the  sample.  Table  3  presents  these  results.  The  results 
indicate  cognitive  ability  positively  predicted  listening  and  reading  achievement  scores  for  individuals 
learning  MSA.  However,  motivation  to  learn  was  not  predictive  of  either  achievement  outcome.  The 
interaction  between  these  two  variables  was  also  not  statistically  significant. 

Table  3.  Regression  of  1AT  Achievement  Test  Scores  onto  Motivation  to  Learn  and  General  Cognitive 
Ability  for  MSA  Trainees  (n  =  147) 


Step  1 _  _ Step  2 


B 

P 

B 

P 

Listening  Achievement 

Motivation 

-.19 

-.03 

-.19 

-.03 

Cognitive  Ability 

1.54** 

.23 

1.56** 

.23 

Motivation  X  CA 

- 

- 

-.37 

-.05 

R2 

.05 

* 

.06* 

A  R1 

- 

<.01 

Reading  Achievement 

Motivation 

.05 

.01 

.05 

.01 

Cognitive  Ability 

1.59** 

.21 

1.60** 

.22 

Motivation  X  CA 

- 

- 

-.25 

-.03 

R2 

.05 

* 

.05 

* 

A  R2 

- 

<.01 

Note.  CA  =  Cognitive  ability.  B  =  unstandardized  coefficient,  p  =  standardized  coefficient.  Cognitive  Ability  and  Motivation  to 
Learn  were  standardized  for  this  analysis. 

*  p<  .05,  **  p<  .01. 


Findings — French  Trainees  Only 

We  next  evaluated  the  relationships  of  interest  for  French  trainees  (n  =  115).  French  represented  the 
second  largest  training  language  in  the  sample.  Table  4  presents  these  results.  The  results  indicate 
cognitive  ability  positively  predicted  listening  and  reading  achievement  scores  for  individuals  learning 
French.  Motivation  to  learn  was  not  predictive  of  either  achievement  outcome.  The  interaction  between 
these  two  variables  was  also  not  statistically  significant. 
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Table  4.  Regression  of  IAT  Achievement  Test  Scores  onto  Motivation  to  Learn  and  General  Cognitive 
Ability  for  French  Trainees  (n  =115) 


Step  1 

Step  2 

B 

p 

B 

p 

Listening  Achievement 

Motivation 

1.15 

.14 

1.11 

.13 

Cognitive  Ability 

3.03** 

.36 

3.06** 

.36 

Motivation  X  CA 

- 

- 

.22 

.03 

R2 

.15** 

.15** 

A  R1 

- 

<.01 

Reading  Achievement 

Motivation 

.07 

.01 

.09 

.02 

Cognitive  Ability 

2.77** 

.52 

2.75** 

.51 

Motivation  X  CA 

- 

- 

-.15 

-.03 

R2 

27** 

27** 

A  R2 

- 

<.01 

Note.  CA  =  Cognitive  ability.  B  =  unstandardized  coefficient,  p  =  standardized  coefficient.  Cognitive  Ability  and  Motivation  to 
Learn  were  standardized  for  this  analysis. 

* p<  .05,  **p<. 01. 


Conclusions 

The  objective  of  this  report  is  to  document  the  relationships  between  SOF  trainees’  general  cognitive 
ability,  motivation  to  learn,  and  achievement  during  IAT.  The  findings  point  to  several  conclusions: 

•  General  cognitive  ability  showed  a  robust  positive  relationship  with  achievement  across  all 
analyses. 

o  This  finding  suggests  trainee  AFQT  scores  may  be  useful  for  language  placement,  such 
that  higher  ability  learners  could  be  identified  and  placed  into  more  difficult  training 
languages.  Placing  trainees  with  greater  learning  potential  into  more  difficult  languages 
could  improve  learning  outcomes  (e.g.,  skill  acquisition)  and  training  success  in  those 
languages. 

•  Motivation  to  learn  showed  an  inconsistent  relationship  with  achievement.  After  accounting  for 
training  language,  more  highly  motivated  trainees  showed  greater  listening  achievement  than  less 
motivated  trainees.  This  relationship  was  not  found  for  reading  achievement. 

o  This  pattern  of  findings  suggests  trainee  motivation  (assessed  prior  to  training)  may  not 
be  a  reliable  indicator  of  achievement  in  the  IAT  training  context  examined  in  this  report. 

•  The  relationships  between  cognitive  ability,  motivation  to  learn,  and  achievement  were  largely 
independent.  That  is,  the  relationship  between  cognitive  ability  and  achievement  did  not  appear  to 
vary  as  a  function  of  trainee  motivation  to  learn.  Similarly,  the  influence  of  motivation  to  learn 
did  not  appear  to  vary  as  a  function  of  trainees’  cognitive  ability. 

o  This  finding  suggests  the  potential  positive  influences  of  cognitive  ability  and  motivation 
to  learn  in  this  learning  context  are  additive  (as  opposed  to  multiplicative). 
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APPENDIX  A 


Means,  Standard  Deviations,  and  Zero-Order  Correlations  for  Study  Variables 


Variable 

Mean 

Std.  Dev. 

1 

2  3  4 

1 .  Motivation  to  Learn 

5.71 

0.86 

- 

2.  Cognitive  Ability  (AFQT) 

227.74 

17.82 

.06 

3.  Overall  Listening  Achievement 

86.66 

7.51 

14** 

.26** 

4.  Overall  Reading  Achievement 

88.46 

6.28 

12** 

24**  57** 

Note,  n  =  550. 
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ABOUT  SWA  CONSULTING  INC. 

SWA  Consulting  Inc.  (formerly  Surface,  Ward,  and  Associates)  provides  analytics  and  evidence-based 
solutions  for  clients  using  the  principles  and  methods  of  industrial/organizational  (I/O)  psychology.  Since 
1997,  SWA  has  advised  and  assisted  corporate,  non-profit  and  governmental  clients  on: 

•  Training  and  development 

•  Performance  measurement  and  management 

•  Organizational  effectiveness 

•  Test  development  and  validation 

•  Program/training  evaluation 

•  Work/job  analysis 

•  Needs  assessment 

•  Selection  system  design 

•  Study  and  analysis  related  to  human  capital  issues 

•  Metric  development  and  data  collection 

•  Advanced  data  analysis 


One  specific  practice  area  is  analytics,  research,  and  consulting  on  foreign  language  and  culture  in  work 
contexts.  In  this  area,  SWA  has  conducted  numerous  projects,  including  language  assessment  validation 
and  psychometric  research;  evaluations  of  language  training,  training  tools,  and  job  aids;  language  and 
culture  focused  needs  assessments  and  job  analysis;  and  advanced  analysis  of  language  research  data. 

Based  in  Raleigh,  NC,  and  led  by  Drs.  Eric  A.  Surface  and  Stephen  J.  Ward,  SWA  now  employs  close  to 
twenty  I/O  professionals  at  the  masters  and  PhD  levels.  SWA  professionals  are  committed  to  providing 
clients  the  best  data  and  analysis  upon  which  to  make  evidence-based  decisions.  Taking  a  scientist- 
practitioner  perspective,  SWA  professionals  conduct  model-based,  evidence-driven  research  and 
consulting  to  provide  the  best  answers  and  solutions  to  enhance  our  clients’  mission  and  business 
objectives.  SWA  has  competencies  in  measurement,  data  collection,  analytics,  data  modeling,  systematic 
reviews,  validation,  and  evaluation. 

For  more  information  about  SWA,  our  projects,  and  our  capabilities,  please  visit  our  website  (www.swa- 
consulting.com)  or  contact  Dr.  Eric  A.  Surface  (esurface@swa-consulting.com)  or  Dr.  Stephen  J.  Ward 
(sward@swa-consulting.com). 
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